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Abstract 

Asymptotic  linear  prediction  and  interpolation,  for  statistically  contaminated 
vector  stationary  processes  is  considered.  Both  prediction  and  interpolation  are 
then  formalized  as  stochastic  games  with  saddle  point  solutions.  The  existence  of 
unique  solutions  on  convex  and  closed  classes  of  vector  stationary  processes  is 
shown.  Then,  those  solutions  are  found  and  analyzed,  for  two  specific  classes  of 
vector  stationary  processes. 
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1. 


Introduction 


The  prediction  and  interpolation  problems  for  stationary  processes  have  received 
conoid^i-.: oie  attention  for  a  number  of  years.  The  bulk  of  the  work  concentrates 
around  scalar  stationary  processes  and  the  parametric  model.  The  assumption  there 
is  that  the  measure  of  the  stationary  process  is  well  known.  The  initial  significant 
results  on  prediction  and  interpolation  for  the  parametric  model  were  given  by 
Kolmogorov  [2]  and  Wiener  [1],  Most  of  the  to  date  results  on  the  same  model  can 
be  found  in  Hannan  [12].  There,  both  scalar  and  vector  stationary  processes  are 
considered . 

Strictly  speaking,  the  term  prediction  refers  to  the  extraction  of  a  datum 
from  the  process,  when  a  number  of  past  process  data  have  been  observed  noiselessly. 
The  term  interpolation  refers  to  the  same  extraction,  when  past  as  well  as  future 
noiseless  process  data  are  available.  The  two  terms  are  extended  sometimes  to 
include  noisy  observation  data.  Some  results  on  those  extended  problems,  and  for 
the  parametric  model,  can  be  found  in  [4]  and  [6].  We  point  out  here  that  the 
majority  of  studies  on  the  extended  problems  consider  asymptotic  and  linear  predic¬ 
tion  and  interpolation  operations. 

The  last  few  years  considerable  attention  has  been  given  to  the  robust  extended 
prediction  problem.  Little  attention  has  been  given  to  the  robust  nonextended  inter¬ 
polation  problem.  The  robust  model  is  nonparametric,  and  the  assumption  is  that  the 
measure  that  generates  the  stationary  process  is  not  well  known.  The  existing 
work  on  robust  extended  prediction  and  interpolation  concentrates  around  scalar 
stationary  processes,  linear  asymptotic  prediction  and  interpolation  operations, 
and  noisy  observation  data.  Representative  results  here  include  robust  Wiener 
and  Kalman  filtering  for  scalar  stationary  processes,  and  can  be  found  in  [3], 

[5],  [7],  [10],  and  [11].  Related  work  on  time  series  outliers  can  be  found  in 
[8].  The  work  in  [9]  considers  the  robust  nonextended  prediction  problem,  for 
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linearly  contaminated  scalar  stationary  processes.  The  robust  solution  is  found 
there  in  the  class  of  asymptotic  linear  prediction  operations.  In  [13],  a  game 
theoretic  formulation  of  the  robust  extended  prediction  problem  is  presented. 

There,  statistical  contamination  on  the  measure  of  the  processes  is  assumed,  and 
it  is  concluded  that  linear  prediction  operations  may  not  be  robust  unless  pre- 
ceeded  by  appropriate  nonlinear  transformations.  In  [14],  some  performance  bounds 
on  the  robust  extended  prediction  and  interpolation  problems  are  given. 

In  the  present  paper,  we  consider  the  robust  nonextended  prediction  and  inter¬ 
polation  problems  for  vector  stationary  processes.  Vector  processes  have  not  been 
treated  in  the  robust  literature,  and  they  present  some  interesting  peculiarities. 

In  particular,  we  consider  prediction  and  interpolation  in  the  absence  of  noise. 

We  assume  that  the  vector  process  is  a  member  of  a  convex  and  closed  class  of  sta¬ 
tionary  vector  processes.  We  adopt  asymptotic  and  linear  prediction  and  interpola¬ 
tion  operations.  To  maintain  consistency  with  the  results  in  [13],  we  assume  that 
a  nonlinear  stationary  operation  has  preceeded  the  linear  prediction  and  interpola¬ 
tion  operations.  As  exhibited  in  [13],  an  appropriate  nonlinear  stationary  operation 
maps  the  convex  and  closed  class  of  vector  stationary  processes  onto  another  convex 
and  closed  class  of  such  processes.  We  formulate  the  prediction  and  interpolation 
problems  as  stochastic  games  with  saddle  point  solutions.  We  find  these  solutions 
explicitly,  for  two  convex  and  closed  classes  of  vector  stationary  processes.  One 
of  the  classes  represents  linear  contamination  of  a  nominal  vector  process.  The 
other  class  includes  vector  processes  with  fixed  energy  on  prespecified  frequency 
quantiles . 

The  organization  of  the  paper  is  as  follows.  In  section  2,  we  formulate  the 
prediction  and  interpolation  games  and  we  prove  the  existence  of  unique  solutions. 

In  section  3,  we  analyze  the  above  games  for  the  class  representing  linear  contamina¬ 
tion  of  a  nominal  vector  process.  In  section  4,  we  analyze  the  prediction  and  inter¬ 
polation  games  for  the  class  of  vector  processes  with  fixed  energy  on  prespecified 
frequency  quantiles.  In  section  5,  we  present  some  conclusions. 
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2.  Preliminaries 

We  consider  the  prediction  and  interpolation  problems  for  stationary  vector 
processes.  We  assume  that  the  statistical  structure  of  the  vector  process  is 
incompletely  known.  We  model  this  incomplete  knowledge  by  a  convex  and  closed 
family  F  of  stationary  vector  processes.  In  the  prediction  problem,  we  adopt 
the  class  of  one-step,  asymptotic  linear  predictors.  In  the  interpolation 
problem,  we  adopt  the  class  of  linear  interpolators  that  operate  on  the  infinite 
past  and  the  infinite  future  data  sequences  from  the  vector  stationary  process.  We 
select  the  mean  square  error  as  the  payoff  function  and  we  formalize  a  saddle  point 
game  on  FxS^  and  FxS^  respectively.  We  call  the  corresponding  solutions  of  the 
game^ robust.  We  should  point  out  here  that  our  solutions  may  not  satisfy  the  pro¬ 
perties  of  qualitative  robustness.  Indeed,  as  exhibited  in  [13],  linear  operations 
on  a  stationary  process  may  not  satisfy  the  sufficient  conditions  for  qualitative 
robustness.  These  sufficient  conditions  will  be  satisfied,  however,  if  an  appropriate 
nonlinear  operation  preceeds  the  linear  predictor  or  interpolator  [13].  Thus,  in 
this  paper,  we  will  assume  that  the  class  F  of  vector  stationary  processes  is,  in 
general,  induced  by  an  appropriate  stationary  nonlinear  operation  on  the  original 

class  F  of  such  processes.  We  will  first  formalize  the  games  on  FxS  and  FxS  for 
o  pi 

an  arbitrary  convex  and  closed  set  F.  Then,  we  will  consider  some  specific  F 
choices. 

Let  F  be  some  convex  and  closed  family  of  discrete-time,  n-dimensional, 
complex-valued,  vector  stationary  processes.  Let  y  denote  the  measure  of  some 
process  in  F,  and  let  F^(u)  denote  the  spectral  distribution  matrix  of  the  process 
y  in  F.  The  spectral  distribution  matrix  F^(u>)  is  defined  on  [~7T,7r],  it  has 


Hermitian  nonnegative  increments  F^(u)^)-F^ (u^)  ;  —  W2*  *s  contlnuous  from 

the  right,  and  it  satisfies  the  equation  F^(-ir)  =  ().  Furthermore,  f^(uj)  defines 
a  finite  matrix  measure  on  the  Borel  field  of  the  measurable  space  [-t^tt],  in 


the  sense  that  j  j  F^(  [  — tt  ,  tt ^ )  j  |  *>  I  |f^(tt)  |  |  <  00 ,  under  any  matrix  norm  ||*|j.  Through¬ 
out  this  paper,  we  will  assume  that  the  above  finite  matrix  measures  are  absolutely 

continuous  with  respect  to  the  Lebesque  measure  on  [-tt.tt],  for  all  y  in  F.  Then,  for 

any  u  in  F,  the  spectral  density  matrix  ^(w)  =  ~  F^(uj)  exists,  and  it  is,  in  general, 

a  Hermitian  nonnegative  definite  matrix  whose  elements  are  Lebesque  integrable  func¬ 
tions  on  the  measurable  space  ( [  — TT ,  tt  ] ,  B  ). 

Let  be  the  space  of  all  nxn  complex  matrices  A(w)  with  the  property: 


A(oo)  e  tr 


A  (<d)  f^((u)  A  (id)  did  <  «> 


;  where  y  some  stationary  process  in  F,  tr  means  trace,  and  *  stands  for  conjugate. 

Then,  is  a  Hilbert  space  with  inner  product  and  norm  given  respectively  by 
the  following  expressions: 


(A^a)),  A2(w))  =  tr  \  A2  (to)  f^ (to)  A2(to)  dto  ;  A-^id),  A2(to)  £  (2) 


||A1(C0)I|S  £  (A1(to),  A^to))172 

y 

oo 

Let  S  and  be  the  sets  of  all  matrix  polynomials  of  the  form  ^ ^  A^  e^ 


^ ^  A^  e^w  respectively;  where  { A^}  is  any  square  summable  sequence  of  constant 


nxn  matrices.  It  is  easily  seen  that  S  d  S  and  S.d  S  ;  V  y  e  F.  We  will  denote 

p  y  i  y 

by  members  of  the  set  S^.  We  will  denote  by  members  of  the  set  S^. 

Let  I  be  the  identity  n-dimensional  matrix.  Then,  I  eS  ;  V  u  e  F.  Given  y  e  F, 
n  n  y 

it  is  well-known  from  the  classical  theory  of  mean  square  linear  prediction  and  inter¬ 
polation  [12],  that  in  the  frequency  domain  both  problems  correspond  to  appropriate 
projections  of  the  identity  matrix  In  particular,  in  the  prediction  problem  the 
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projection  of  onto  the  subspace  is  sought.  In  the  interpolation  problem, 

the  projection  of  In  onto  the  subspace  is  sought,  instead.  For  given  y  in 

F,  the  above  projections  are  realized  by  two  infima  inf  e  (y,e  )  and  inf  e  (y,g  ) 

„  c-C  P  1?  „  c-o  i  T. 


respectively,  where: 


V*P  p  "  Vs! 


epCy.Sp)  £  tr  1  (i^Cw))*  ff/uO  (i^Co))  do  ;  ^  e  S? 


(TT 

(I- 

-IT 


■^(w))  f^(w)  do  ;  e  S, 


It  is  also  well-known  [15]  that: 


ep<u) '  *"!.  V^-V  " V^V 

^P  P  ^P  P 


j  n  ) 

exp<(2im)  1  tr [£og  2tt  f^(o)]  dw);  p  e  F 

(  J-7T  ) 


eA(y)  =  inf  =  min  e1(P»£.i) 

&iCSi  &iCSi 


2  rf  -i  i-1 

4ir  trl  f^(o)  do  ;  y 
J-TT 


We  now  consider  games  on  FxS  and  FxS^,  with  payoff  functions  given  respectively 

“ft  jit 

by  e^Cy.jJp)  in  (4),  and  e^(y,j>^)  in  (5).  We  are  seeking  pairs  (y^.g^)  on  FxS^  and 
(y*,£*)  on  FxS such  that: 

V  *  £  F  5  1  ep(lJp’-4)  ±  ep(yp’V  5  V  £p  e  Sp  (8) 

V  y  e  F  ;  ei(y,£*0  <  e^yJ.^J)  <  e1(y*,£j.)  ;  V  gi  e  S±  (9) 

If  the  pairs  in  (8)  and  (9)  exist,  we  call  them  solutions  of  the  corresponding 
games.  We  proceed  with  a  theorem  whose  proof  is  in  the  appendix. 
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Theoreg  1 

Let  F  he  a  convex  and  closed  family  of  n-dimensional  vector  stationary  processes 
with  spectral  distribution  matrices  satisfying  the  properties  stated  in  this  section. 
Let  ep(p)  and  e^p)  be  given  by  (6)  and  (7)  respectively.  Then,  there  exist  unique 
measures  Pp  and  p^  in  F,  such  that: 


e  (p*)  =  sup  e  (p) 

p  p  peF  p 


X 

e. (p.)  =  sup  e  (p) 

11  peF  1 


Let 


e  S.  be  such  that: 
x 


(10) 

(11) 


e 

P 


(p*,£*)  =  e  (p 
P  P 


*) 

P 


•iK*i>  - 


Then,  the  pairs  (p^j^)  and  (p*,^*)  are  the  unique  solutions  of  the  games  in 
(8)  and  (9)  respectively. 


Theorem  1  and  its  proof  are  parallel  to  theorem  1  in  [14],  In  the  latter,  more 
general  asymptotic  as  well  as  nonasymptotic  filters  and  smoothers  are  included. 
According  to  the  above  theorem,  to  find  solutions  for  the  games  in  (8)  and  (9),  it 
is  sufficient  to  search  for  measures  that  satisfy  the  suprema  in  (10)  and  (11) . 

To  this  point,  we  considered  arbitrary  convex  and  closed  families  F  of  stationary 
vector  processes.  To  find  explicit  solutions  for  the  games  in  theorem  1,  we  will 
adopt  two  specific  such  families.  Since  in  the  error  expressions  in  (4)  and  (5),  the 
measure  of  the  vector  process  appears  through  its  spectral  density  matrix  only,  we 
will  define  the  two  families  through  their  spectral  characteristics. 

The  first  family  we  consider  satisfies  the  general  properties  stated  in  the 
beginning  of  this  section,  and  it  corresponds  to  linear  contamination  of  a  nominal 


7 


measure  y^.  In  particular,  denoting  by  the  well-known  spectral  density  matrix 

of  the  measure  y^,  we  define  this  family  in  the  frequency  domain  as  follows: 


(A)  F  =  f(uO  :  jE(uj)  =  Cl-e)  f»(u)>  +  e  hC&O  ;  a)  el-7r,7r] 


;  where  e  given  and  such  that:  0  <  e  <  1, 

f^Cw)  well-known  positive  definite  Hermitian  matrix, 

h(w)  nonnegative  definite  Hermitian  matrix  satisfying  the  energy 

-1  f11  I 

constraint:  (2tt)  tr  I  h(uj)  dui  <  W,  for  given  positive  W.j 


The  second  family  we  consider  satisfies  again  the  general  properties  in  the 
beginning  of  this  section,  and  it  corresponds  to  fixed  energy  within  a  finite  number 
of  subsets  of  the  Borel  field  B^.  Specifically,  this  family  is  defined  as  follows: 


(B)  Fq  =  j  f(w)  :  tr  I  f(u>)  du)  =  c  ,  ;  i=l,2,...,k,  tr  |  f.(^)  d<j>  =  c 


;  where  A±  e  B^  ;  Vi,  A±0  A  =  $  ;  V  i  f  j  ,  \ J  A±CT  [  — tt , tt ] , 


>  }  ,  c  and  f(oj)  positive  definite  Hermitian  matrix 

**  i 


Given  two  nxn  spectral  density  matrices  f^(to)  and  f^Cw),  define  the  metric 
n 

y]  |A<»<«)  -  A^(o>)|;  where  {Ap^(u));  1  <  i  <  n}  the  eigenvalues  of  the  matrix 

1=1 

f .  (oj)  ;  j=l,2.  Then,  both  families  FT  and  F_  are  clearly  convex  and  closed  with 

respect  to  that  metric;  thus  they  satisfy  the  conditions  in  theorem  1.  Therefore, 

it  is  sufficient  to  search  for  the  suprema  sup  e  (y) ,  sup  e  (y) ,  sup  e  (y) , 

P  1-^q  “  «FL>e  1 

sup  e  (y) . 

peF, 
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In  section  3,  we  will  solve  the  prediction  and  interp.  '.ation  games  for  the  class 

F  .  In  section  4,  we  will  solve  the  same  games  for  the  class  F  . 

L»  -  Q 

3.  The  Solution  of  the  Games  for  F_ 

- XJ>e 

Due  to  the  results  in  theorem  1,  the  solution  of  the  prediction  and  the  inter¬ 
polation  games  corresponds  respectively  to  the  measures  that  realize  the  suprema  of 
the  errors  e  (y)  and  e.(i-0  in  (6)  and  (7).  In  this  section,  we  are  searching  for  the 
above  suprema  on  F,  ;  where  the  class  FT  is  given  by  (A)  in  section  2. 

Given  some  spectral  density  matrix  f(u>)  in  F  let  us  denote  by  (X.(co)  ; 

L>e  J 

j=l,...,n}  its  eigenvalues.  Since,  fCco)  is  positive  definite,  X  (to)  >_  0;  V  j, 

V  deI-it,  it]  .  Let  us  denote  by  {X?(w);  j=l,...,n}  the  eigenvalues  of  the  nominal 

spectral  density  matrix  .  Let  us  define  two  scalar  functions  on  [  — tt , tt ] ,  that 

we  will  use  later  in  our  derivations.  In  particular,  if  (X.((d);  1  <  j  <  n}  are  the 

eigenvalues  of  the  spectral  density  matrix  £(u))  in  F  ,  we  define: 

L|  0 

X  .  (u)  =  min  X.(w)  ;  w  E  [  — tt , tt ]  (12) 

min  ,  . . .  1 

l<j<n  J 

X°  (w)  «  max  X?(uO  ;  w  e  [-ir,ir]  (13) 

maX  l<j<n  J 

At  this  point  we  observe  that  for  arbitrary  matrix  A  with  eigenvalues  {X^},  we 
have  the  known  identities:  tr{£og  A}  =  £og{det  A)  and  det  A  =  H  X  ;  where  det  denotes 

j 

determinant.  If  we  apply  the  above  identities  to  the  error  expression  e^(y)  in  (6), 
we  obtain: 


e  (y)  =  e  (f) 
P  P  _ 


2tt  exp  J  (27rn)  £og  (  H  X  (u>))  do) 

'  j-1  J  ' 

_ TT  J 


;  where  {X^  (w)  ;  1  <  j  <  n}  the  eigenvalues  of  the  spectral  density  matrix  fOo)  that 
corresponds  to  the  measure  y. 
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For  completion,  we  reexpress  the  interpolation  error  e^(y)  in  (7): 


e  (n)  =  e  (f )  =  4tt2  tr| 
i  i  — 


r 


r  tt 


dco 


-IT 


-1 


(15) 


From  expression  (14),  we  observe  that  the  prediction  error  e^(f)  is  solely  a 

function  of  the  eigenvalues  of  the  matrix  f^(to)  .  The  eigenvectors  of  f(w)  do  not 

appear  explicitly  in  the  error  expression  ep(f).  At  the  same  time,  there  are  no 

explicit  statements  in  the  description  of  the  class  F  in  (A) ,  referring  to  the 

L,£ 

eigenvectors  of  the  spectral  density  matrices  _f(w) .  For  those  reasons,  we  will  take 
two  extreme  directions.  We  will  first  assume  that  the  class  Tx  includes  spectral 
densities  with  no  restrictions  on  the  eigenvectors,  besides  the  ones  implied  by  the 
semipositive  definite  requirement  on  the  matrices  h.('->)  .  Then,  we  will  consider  a 
subclass  FJ  contained  in  F  .  The  subclass  F’  will  contain  only  those  matrices 

li  y  W  1j  |  L  |  C 

_f(oj)  in  F  that  have  eigenvectors  identical  with  those  of  the  nominal  spectral 
density  matrix  ^(tn)  •  That  is,  the  spectral  density  matrices  in  F^  ^  will  have 
identical  projections  on  the  directions  determined  by  the  eigenvectors  of  the  matrix 
^(w)  .  We  observe  from  expression  (15)  that  the  interpolation  error  e^(f )  is,  in 
general,  a  function  of  both  the  eigenvalues  and  the  eigenvectors  of  the  spectral 
density  matrix  f(u>) .  Thus,  we  will  find  a  suboptimal  solution  of  the  interpolation 
game  on  F  .  We  will  solve  the  prediction  game  for  both  the  classes  F,  and  F'  . 
We  first  state  formally  the  description  of  the  class  F’  . 

(C)  F*  =  jf(co)  :  f (03)  e  F  ,  and  f(^)  has  eigenvectors  identical  with  those 

L>y£  J  L  ,£ 


I 


of  the  matrix  f^w). 


We  will  first  find  the  solution  of  the  prediction  game  (or  equivalently  the 
supremum  of  ep(f))  within  the  class  F L  £ .  To  do  that  we  will  use  a  trick.  We  will 
solve  the  corresponding  optimization  problem  within  some  subclass  F  of  F  .  Then, 
we  will  show  that  the  so  obtained  solution  is  sufficient,  in  the  sense  that  it  also 


10 


solves  the  optimization  problem  in  FT  .  We  will  first  define  the  subclass  F  and 

Ll  y  Lr 

we  will  prove  some  of  its  properties. 


(D)  r  =  /  f  (to)  :  f(tu)  e  F  ,  and  >.  .  (to)  >  (l-s)  A°  (to)  ;  V  to  z  [  -tt , tt ] ; 
I —  —  L,E  mm  —  max 

<  where  A^^Cto)  and  A°ax(to)  are  given  by  expressions  (12) 

(  and  (13) 


Proposition  1 

The  class  F  is  nonempty,  convex  and  closed,  and  it  is  contained  in  F^ 

Proof 

i)  Let  f(<o)  be  some  spectral  density  matrix  in  F^  £.  Then,  due  to  the  non¬ 
negative  definite  requirement  on  h(to)  ,  we  have  that  the  matrix  f^(tu)  - 
(l-e)f^ (to)  is  nonnegative  definite.  Let  X(to)  be  the  eigenvector  of  f^(to) 

that  achieves  A°  (to)  .  Due  to  the  complete  freedom  of  the  eigenvectors 

T 

in  Ft  _,  there  exists  some  f  (oj)  in  it,  such  that  X  (to) f  (u)X(to)  =  A  .  (to). 

—  min 


Thus,  there  exists  some  f (to)  in  Fr  such  that: 

L*  y 


XT(<o)f (to)x(co)  -  (l-e)XT(oj)f/,((o)X(to)  =  A  .  (to)  -  (1-e)  A°  (to)  >  0 
—  — u  ram  max  — 

and  nonemptiness  is  established. 

ii)  Let  f^^  (to)  and  _f^(to)  be  in  F.  Let  a  be  some  constant  in  (0,1).  Then, 

[a  f(1)(oo)  +  (1-a)  f(2)  (to)]  e  FT  .  Let  X(to)  be  the  eigenvector  that 
achieves  the  Amin  a  f(l)  +  f(2)(w)  for  the  matrix  a  (oo)  + 

(l-a)f(2)(to).  Then, 

^min  af^+(l-a)f  (2)^“  XT(to)  [af  ^  (to)  +  (l-a)_f^  (to)  ]  X(co)  =  a  XT(to)  .f  ^  (to)  X(to)  + 
+  (1-a)  XT(ao)  f^2\to)  X(co)  >  a  Amin  f(i)(to)  +  (1-a)  Amin  f(2)(to)  >  (1-e)  A°&x(to) 


Thus,  convexity  has  been  proven.  Closeness  is  straightforward  due  to  the 

inclusion  of  equality  in  A  .  (to)  >  (1-e)  A°  (to). 

min  —  max 
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lii)  Let  f (j)  £  F.  Let  X(co)  any  vector.  Then, 


XT(uj)  f(w)  X(u)  XT(w)  f.(oj)  X(oi) 

- — = - >  X  .  («)  >  (l-£)  X°  (u)  >  (1-e)  - -=2 - 

XT(w)  X(u>)  ”in  max  XT(oj)  X(oj) 


X  (<d)  f(w)  X(to)  X  (oj)  f^(to)  X((j) 

- - - (1-e)  - - -  >  0  ;  V  X(to) 

X  (to)  X(u>)  X  (to)  X(to) 


Thus,  f(to)  £  F  ,  and  F d.  F  . 

L  i  c  L4  ^  C 


Due  to  proposition  1,  the  class  F  in  (D)  satisfies  the  properties  in  theorem  1. 

Thus,  the  games  in  (8)  and  (9),  on  FxS^  and  FxS^  respectively  are  equivalent  to 

establishing  the  suprema  of  e  (f)  and  e,(f)  on  F.  Since  F  is  contained  in  F  ,  the 

p  1  L ,  G 

-1  r 

energy  constraint  (2tt)  tr  1  h(u>)  dw  <  W  is  also  implied  in  F.  Writing 

J-TT 


e  h('a>)  =  f_0o)  -  (1-e)  f^(to)  »  and  tr  I  f_ (to)  dcj  = 


’Tr  n 


[E  aiH 

4=i  J 


da),  we  obtain  the 


following  form  of  the  above  energy  constraint: 


r 

r  J1 ,  I 

fir  n 

p  "1 

Y  X((j)  dto  <  (l-£) 

[  y  x°(u)) 

\ 

1  J 

Z-r  x  I 

J-TT 

Li=i  J 

-Tr  L  i=l  J 

The  maximization  of  the  errors  e  (f)  and  e,(f)  on  F  reduces  then  to  two  optimi- 

p  -  i  - 

zation  problems  with  two  inequality  constraints  on  just  the  eigenvalues  of  the  matrices 
f\'S)  .  The  optimization  problems  have  unique  solutions,  due  to  theorem  1.  Using  the 
expressions  in  (14)  and  (16),  and  the  description  of  F  in  (D),  we  first  state  the 
optimization  problem  for  e^(f)  on  F  and  its  solution  in  a  lemma. 
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Lemma  1 


The  supremum  of  e  (f)  on  F  in  (D)  corresponds  to  the  solution  of  the  following 


constraint  optimization  problem: 


sup  ]  £0g 


n 

[n  xjM] 


subject  to: 


j  []£  xi(u)] da)-  a~s)  J  [2  xi(u)] 


du  +  e  2m  w 


^ j  *  (1-e)  Xmax(u)  ;  v  j  ,  v  to  e  [  —it , tt ] 

;  where  X°ax((o)  is  given  by  (13)  . 

The  unique  solution  of  the  above  optimization  problem  is  represented  by  an 

*/|f  ^ 

n-tuple  .X^(u>) , . . .  ,X^((d) )  of  eigenvalues;  where: 


Vj  ;  X*(u))  =  X*(oj)  = 


Xlv^  »  W:  0--c)  X°  (u>)  >  y 

max  m  a-v  * 


;  0):  (1-e)  X°  (00)  <  y 

max  —  * 


and  Y  is  a  constant  uniquely  defined. 


We  observe  that  the  solution  in  the  lemma  does  not  involve  eigenvectors  at 
an*  Slnce  there  are  n°  eigenvector  restrictions  in  the  class  F,  the  implication 
is  that  the  problem  is  solved  by  any  spectral  density  matrix  whose  eigenvalues  are 
given  by  (17)  and  whose  eigenvectors  are  unrestricted.  Thus,  the  uniqueness  of 
the  solution  refers  to  eigenvalues  only.  We  point  out  the  similarities  of  the 
solution  in  (17)  with  Hosoya’s  solution  [11],  for  scalar  stationary  processes.  We 
exhibit  this  solution  graphically,  in  figure  1.  We  now  present  the  proof  of  the  lemma. 
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Proof  of  Lemma  1 

To  find  the  solution  of  the  optimization  problem  in  lemma  1,  we  write  the  Euler- 
Langrange  equations  and  apply  the  Kuhn-Tucker  conditions: 


X“l(u))  -  y  1  -  IK  (  a)  =0  ;  1  <  i  <  n 
y  (to)[X  (to)  -  Cl-e)  X°  (to)]  =  0  ;  1  <  i  <  n 

i  1  max  “ 

;  where  y^Cw)  ;  1  <  i  <  n  the  Euler-Langrange  multipliers. 

From  the  Euler-Langrange  equations  we  find  that  the  optimal  solution  satisfies  the 
condition: 


*  * 

X  ft*  =  X  (to)  ;  1  <  j  <  n 

*  o 

;  where  X  (u>)  is  equal  to  either  (1-e)  ^max(u)  or  to  a  constant  y  such  that 

y  >  (l-£)  X  (to)  .  The  constant  y,  as  well  as  the  regions  E  and  E*"  on  which  X*(oo) 
max  y  y 

is  respectively  equal  to  (1-e)  X°  (to)  or  to  Y  are  determined  by  the  following 

max 

equation: 


X  (to)  dto  =  (1-e) 


X°  (to)  dto  +  y 

max 


dto  = 


n t 


-l 


ti-o  E 

i=l  J- 7T 


X°(u>) 


dto  +  e  2tt  w  j 


(18) 


We  wish  to  prove  that  (18)  has  a  unique  solution  with  respect  to  y  and  E^.  Let 
us  first  define  for  simplicity  in  notation: 


J 


X  (to)  dto  =  V 
max 


(19) 
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-ll 


u-o  Z 


L 


1=»1  J-TT 


da)  4*  e  2tt  W  j  = 


(20) 


Substituting  expressions  (19)  and  (20)  in  expression  (18),  we  obtain: 


cx(y)  =  |  lY  -  (1-e)  X°ax((jj)]  du  -  Q  -  (1-e)  V 


(21) 


It  is  easily  shown  that  <x(y)  is  monotonically  nondecreasing  with  y,  taking 
values  from  0  to  Also,  Q  -  (1-e)  V  >  0.  Therefore,  the  equation  in  (21)  is 
satisfied  for  a  unique  positive  Y  value.  The  proof  is  now  complete. 


We  will  now  complete  the  search  for  the  solution  of  the  prediction  game  in  (8) 
on  F^  ^.xS^,  by  showing  that  it  is  sufficient  to  place  the  game  on  FxS^  instead; 
where  F  is  given  by  (D) .  We  express  this  result  in  a  theorem. 

Theorem  2 

The  solution  of  the  prediction  game  on  FxS^,  given  by  expression  (17)  in  lemma 
1,  is  also  the  solution  of  the  prediction  game  on  F  xS  ;  where  F  is  described 

LyC  p 

by  (A) . 

Proof 

Due  to  the  established  uniqueness  of  the  solution  on  F  xS  (by  theorem  1)  as 

well  as  the  solution  on  FxS  ,  it  suffices  to  show  that  the  prediction  error  e  (f) , 

P  P  ~ 

for  some  f_  in  F  -  F,  is  bounded  from  above  by  the  prediction  error  e  (f' ),  for 
"  L ,  £  p 

L,e  ’  — *  '  Lt£  '  L.e’ 


some  f'  in  F.  We  define  FT  -  F  such  that  (F  -  F)^  F  =  F.  ,  and 


Oi.c  -nnr- 

Let  f  E  (F,  -  F) .  Then,  X  .  .(to)  <  (1-e)  X  (to) ,  and  there  exists  at 

—  L,e  *  min,£  max 

least  one  eigenvalue  of  f(w)  such  that  it  is  less  than  (1-e)  X°  (to)  for  some  to 

—  max 
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values.  Let  the  eigenvalues  Xj  (to)  , . . .  ,X^(a))  ;  k  <  n  have  this  quality.  For  the 
eigenvalue  X .  (-j)  ;  j  <  k  let  us  create  a  new  eigenvalue  X*.  (to)  by  replacing  X^  (ui)  by 
(1-e)  X°  (u:)  on  those  cu  values  where  X.(u)  is  less  than  (1-e)  X°  (co)  .  Let  us 

alaJrl  J  K1HX 

maintain  the  eigenvalues  {X^(ij)  ;  i  >  k}  unchanged.  The  so  created  n-tuple 
(X|('~),...,  X^(oo))  of  eigenvalues  clearly  satisfies  the  following  inequality  (due  to 
the  nonotonicity  of  the  logorithmic  function): 


n 

log  l  II  X  (to)  ]  do)  < 
i=l  1 


‘IT 


n 

log  [  n  X  *  (a))  ]  da) 
i=l  1 


By  construction,  any  spectral  density  matrix  (u>)  with  eigenvalues  the  n-tuple 
(X|(u) , . . . ,X^(w))  is  in  F.  Thus,  the  proof  of  the  theorem  is  complete. 


We  will  complete  the  coverage  of  the  prediction  game  for  this  section  by  finding 
the  solution  on  F’  xS  ;  where  the  class  F’  is  given  by  (C) .  We  will  proceed 

L^c.  p  Li  ^  £ I 

towards  that  direction  by  first  stating  the  properties  of  the  class  F’  in  a 
proposition. 

Proposition  2 

The  class  FJ  is  nonempty,  convex  and  closed,  it  is  contained  in  F.  ,  and 
it  is  such  that: 


f(w)  e  F’  -*•  X.(U))  >  (1-e)  X°(u)  ;  V  i,  V  u  e  [-it, it] 

™  L  j  c.  x  * 

Proof 

The  class  is  clearly  nonempty  since  it  contains  at  least  ^q(uj).  Xt  *s  a^so 

trivially  convex  and  closed  and  contained  in  F,  .  Now,  since  for  each  f(u)  in 

FI  ,  f(;j)  and  f  —  (u>)  have  the  same  eigenvectors,  the  eigenvalues  of  the  difference 
L  9C  —  U 

JFCu)  -  (1-e)  ^(w)  are  given  by  {X^(oj)  -  (1-e)  X°(u>)  ;  1  <  i  <  n} .  Since 


16 


f(u)  -  (1-e)  should  be  nonnegative  definite  for  every  _f  (to)  in  F*  ,  we 

conclude  then  that  it  is  necessary  that  X^(to)  >  (1-e)  X°(to)  ;  V  i,  V  to  e  [  — tt , tt ] . 
The  proof  of  the  proposition  is  now  complete. 


Due  to  proposition  2,  the  class  F*  satisfies  the  necessary  properties  in 

theorem  1.  Thus,  the  prediction  game  in  (8)  on  F'  xS  is  equivalent  to  establishing 

L ,  £  p  ‘ 

the  supremum  of  e^OE)  on  F^  e*  Since  F^  £  is  contained  in  F^  £,  the  energy  constraint 

in  expression  (16)  is  also  implied  in  F*  .  The  maximization  of  the  error  e  (f)  on 

L,E  p 

F'  reduces  then  to  an  optimization  problem  with  two  inequality  constraints; 

L,E 

where  one  of  the  constraints  is  given  by  (16) ,  and  the  other  one  is  given  by  the 

eigenvalue  inequality  in  proposition  2.  We  state  the  above  optimization  problem  for 

e  (f)  on  F'  „  and  its  solution  in  a  theorem. 

P  —  L,e 

Theorem  3 

The  supremum  of  e  (f)  on  F'  in  (C)  corresponds  to  the  solution  of  the  following 
P  L»c 

optimization  problem: 


subject  to: 


rn 

n 

r  -i 

[L  V“>1 

’-rr 

^i=i  J 

(1-E) 


j  - 


n 

=i 

Xi(0))] 

dto 

*  TT 

n 

fz 

X°(to)l 

J-TT 

*-  i=»l 

J 

V  j, 

,  VIDE 

;  where  (X°(to)  ;  1  <  j  <  n}  the  eigenvalues  of  the  matrix  f^On). 

The  unique  solution  of  the  above  optimization  problem  is  represented  by  an  n- 
£  ^ 

tuple  (X^(w) , . . . ,Xn(w))  of  eigenvalues  such  that: 
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X*(u) 

J 


(1-e)  X?(tj)  ;  u  :  (l-£)  X?(u)  >  y 

3  3  ;  1  <  j  <  n 

y  ;  to  :  (1-c)  X°(to)  <  y 


(22) 


The  constant  y  is  uniquely  defined  and  it  is  the  same  for  all  j  values. 

Proof 

The  proof  is  parallel  to  the  proof  of  lemma  1.  The  Kuhn-Tucker  conditions  here 


are: 

X^Cw)  -  y_1  -  y1((o)  =0  ;  1  <  i  <  n 
y.Cw)  IX.  (01)  -  (1-e)  X°(u)J  =  0  ;  1  <  i  <  n 

i  i  l  —  — 

c 

The  constant  y  and  the  corresponding  regions  (E.  E.  ;  1  <  i  <  n)  where 
each  eigenvalue  A^(u))  is  equal  to  y  or  larger  than  it  are  detemined  by  the  equation: 


n 

r 

f  f  -1 

E  a-e, 

X°(to)  dco  +  y  I  du 

i=l  l  J 

E  *4C 

4  v  4 

i,Y 


a-E)  £ 

i«i  j 


Xi(to)  dui  4-  £  2tt  w 


— TT 


(23) 


Defining  Q  as  in  (20)  and 


V±  =  X°(oo)  du 

/-IT 


(24) 


we  can  write  expression  (23)  in  the  following  form: 


S(Y) 


i  J 

1=1  JeIy 


[y  -  (1-E)  X® (w)  1  du  =  n  Q  -  (1-e)  £  VJL  (25) 

i-1 
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As  In  the  proof  of  lemma  1,  the  monotonicity  and  nonnegativeness  of  B(y)  Is 
easily  established.  It  is  also  true  that  the  right  hand  part  of  (25)  is  positive. 
Thus,  the  uniqueness  of  y  that  satisfies  the  equation  in  (25) . 


Me  exhibit  the  solution  in  (22)  graphically,  in  figure  2.  We  will  now  summarize 

our  conclusions  for  the  prediction  game  on  the  class  F  ,  with  some  comments. 

L|C 

Comments 

Since  the  error  expression  e^(j[)  in  (14)  involves  only  the  eigenvalues  of  the 
spectral  density  matrix  f(w),  and  since  the  description  of  the  class  F  in  (A) 
does  not  involve  eigenvectors  explicitly,  we  nay  adopt  several  assumptions  in  terms 
of  the  JF(io)  eigenvectors.  In  this  section  we  took  two  extreme  directions.  We  first 
assumed  that  the  eigenvectors  are  unrestricted.  Then,  we  assumed  that  the  eigenvectors 
remain  fixed  for  the  whole  class.  In  the  first  case,  the  solution  of  the  prediction 
game  is  given  by  theorem  2.  In  the  second  case,  the  solution  is  given  by  theorem  3. 

The  two  cases  can  be  looked  upon  respectively  as  an  upper  and  a  lower  bound  to  the 
prediction  games  with  any  restrictions  on  the  eigenvectors.  The  case  represented  by 
theorem  2  is  the  most  robust  within  the  linear  class  F  ;  where  any  possible 
restrictions  on  the  eigenvectors  are  not  specified.  It  is  also  the  most  pessimistic. 
Theorem  3  represents  the  least  robust  case  within  the  general  linear  model  of  the 
class  in  (A).  The  solution  of  this  case  does  not  safeguard  against  possible  perturba¬ 
tions  of  the  eigenvectors  of  the  matrix  h(u>)  . 


We  now  focus  on  the  interpolation  game  on  F^  £xS^.  As  we  pointed  out  earlier, 
the  search  for  the  solution  of  this  game  is  more  involved.  This  is  so,  because  the 
error  expression  e^(f)  in  (15)  involves  explicitly  both  the  eigenvalues  and  the 
eigenvectors  of  the  spectral  density  matrix  f  (to)  .  For  that  reason,  we  will  create 
a  lower  bound  on  e  (f ) ,  that  will  be  strictly  a  function  of  the  eigenvalues  of  the 
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matrix  _f(oj) .  We  will  find  a  supremum  of  this  lower  bound  on  F  .  We  will  search 
for  this  supremum  by  going  first  through  the  class  F  in  (D) ,  as  we  did  in  the  predic¬ 
tion  game.  This  supremum  will  provide,  in  general,  a  lower  bound  on  the  saddle  value 

of  the  interpolation  game  on  F  xS  .  We  proceed  with  the  following  proposition. 

£  1 

Proposition  3 

The  error  e^(0  in  (15)  is  bounded  from  below  by  the  function 


\<s> 


=  4  IT 


2  2 


r  n 


J-ir 


du) 


E 

1=1 


x/cu) 

i 


-1 


(26) 


;  where  (A^(ui)  ;  1  <  1  <  n}  the  eigenvalues  of  the  spectral  density  matrix  _f  (o>)  . 

The  function  b  (f)  is  strictly  concave  on  F  and  F  and  it  is  equal  to  the 
L  L|£ 

error  e^Cf)  if  and  only  if  X^io)  =  X  (w)  ;  Vi,  j. 

Proof 

-1  2  -1 

We  will  use  the  known  identity  tr  A  >  n  [tr  A]  ;  where  A  an  nxn  positive 
definite  nonsingular  matrix.  The  above  identity  is  satisfied  with  equality  iff  A 
is  the  identity  matrix  tines  a  constant.  Applying  the  above  expression  to  the  error 
e  (f)  ,  we  find: 


e  (f )  =  4  tt  tr 


-TT 


-1  I  1  2 

£_  (u)  du  >  4  it  n 


tr 


1(to)  dcoj 


,  2  2 
4  TT  n 


E 

J 


-1 


bL(f) 


r.  n  n 

du> 

LJ.„  i«i 

p  1 

with  equality  everywhere  iff  I  £-1(u))  du  is  the  identity  matrix  times  a  constant. 

J-TT  _j_ 

But  this  can  only  occur  if  all  the  eigenvalues  of  f_  (u)  are  equal  to  each  other. 
That,  of  course, implies  that  the  eigenvalues  of  f^(w)  are.  equal  to  each  other.  A 
straightforward  observation  allows  us  to  conclude  that  b  (f)  is  strictly  concave 
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in  the  set 


1  <  i  <  n};  thus,  b  (f)  is  strictly  concave  on  F  and  F. 


L,e- 


We  will  now  seek  the  supremum  of  the  function  b  (f)  in  (26)  on  the  class  F 
in  (D) .  Our  approach  can  be  formalized  as  a  constraint  optimization  problem,  as 
in  lemma  1.  We  present  the  constraint  optimization  problem  and  its  solution  in  a 
lemma. 

Lemma  2 

The  supremum  of  the  function  b^(f)  in  (26)  on  F  in  (D)  corresponds  to  the  solu¬ 
tion  of  the  following  constraint  optimization  problem: 


inf 


n r 

do) 


subject  to: 

r  r 

n 

£  xi<“> 

J-TT 

i=l 

X.  (a))  > 

J  ~ 

where  X°  (cj) 
max 
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L  i*l  J 

dw  +  e  2ir  w 


max 


The  unique  solution  of  the  above  optimization  problem  is  identical  to  the 
solution  (17)  in  lemma  1. 

Proof 

A  unique  solution  exists  due  to  the  strict  concavity  of  b^(f),  and  the  convexity 
and  closeness  of  F.  This  unique  solution  can  be  found  through  an  Euler-Langrange 
formalization.  The  Kuhn-Tucker  conditions  are: 

l  -2 , 


-X“  (w)  +  Y  +  V^w)  =  0  ;  1  <  i  £  n 
u  (u))[A,(o>)  -  (1-e)  A°  (u)]  -  0  ;  1  <  i  <  n 

1  1  DldX 
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;  where  y  (tu) ;  1  <  i  <  n  the  Euler-Langrange  multipliers, 
i  —  — 

From  the  above  conditions  we  obtain: 

[-X~2(uO  +  y]  [X±(u)  -  (1-e)  X^Cw)]  =  0  ;  V  i 

and  thus  the  result  in  the  lemma.  The  uniqueness  of  the  constant  y  is  exactly  as 
in  lemma  1. 


We  now  proceed  with  a  theorem  to  exhibit  the  sufficiency  of  the  solution  in 

lemma  2,  for  the  class  F  as  well.  The  theorem  is  parallel  to  theorem  2  in  the 

L,e 

prediction  game. 

Theorem  4 

The  supremum  of  h^Cf)  on  F ,  given  by  lemma  2,  is  also  the  supremum  of  b^(f)  on 

F  .  If  X*(w)  denotes  the  solution  in  lemma  2,  then  b  ({*)  =  b  (X*)  =  e  (X*)  =  e  (f*) . 
L ,  £  L*  L  i  i 

Proof 

As  in  the  proof  of  theorem  2,  we  select  some  f  in  (F  -  F) .  For  the  eigen- 

— 

values  X,  (o>)  , . . .  ,X,  (u>)  of  f  such  that  they  are  less  than  (1-e)  X°  (to)  for  some  to 
values,  we  create  new  eigenvalues  X^ (to)  i  1  <  j  <  k,  as  in  the  proof  of  theorem  2. 

We  maintain  the  remaining  eigenvalues  unchanged,  and  we  denote  the  whole  newly 
created  set  of  eigenvalues  {X^  (to)  ;  1  <  j  n}.  Then,  we  have  trivially: 


r-n 


dw 


Xp(to)  < 


i=l 


du) 


-IT 


E 

i=l 


Xi(w) 


Therefore,  for  every  f  in  (F  -  F) ,  there  exists  some  f*  in  F  such  that 

L|t 


bL<£'>  i  bL(i> 


The  proof  is  now  complete,  b^(f^  )  =  e^(Jf  ),  because  the  solution 
in  lemma  2  gives  equal  eigenvalues  and  then  (due  to  proposition  3)  equality  is 


attained . 
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We  will  complete  this  section  with  some  comments  on  the  interpolation  game. 
Comments 

Due  to  the  involvement  of  both  the  eigenvalues  and  the  eigenvectors  of  the 

spectral  matrix  f^tu)  in  the  interpolation  error  e^(f) ,  we  were  unable  to  find  the 

general  solution  of  the  game  in  (9)  on  F  xS  .  Instead,  we  found  a  lower  bound  on 

L»,  £  i 

the  saddle  value  of  this  game.  The  value  of  this  lower  bound  is  given  by  lemma  2 

and  theorem  4.  It  is  important  to  point  out,  however,  that  the  developed  lower 

bound  coincides  with  the  error  e^(f)  for  all  _f  in  F  ,  in  at  least  one  case.  This 

is  the  case  where  the  eigenvectors  of  all  the  spectral  density  matrices  in  F  are 

L,  £ 

constant  (not  functions  of  w) .  This  corresponds  to  the  class  of  spectral  density 
matrices  with  identical  projections  on  n  fixed  directions.  Then  the  lower  bound 
bL(0  and  the  error  e  (f)  coincide  for  all  _f . 

4 .  The  Solution  of  the  Games  for  F^ 

In  this  section  we  consider  the  prediction  and  interpolation  games  on  F  xS 

Q  P 

and  F^xS^  respectively,  where  the  F^  class  is  described  by  (B)  in  section  2.  Due 

to  theorem  1,  the  solutions  of  the  above  games  are  provided  by  the  suprema  of  the 

errors  e^(0  aI*d  e^(f)  in  (14)  and  (15)  respectively,  on  the  convex  and  closed 

class  F  .  We  will  use  the  same  notation  as  in  section  3,  and  we  will  denote  by 

(X^(uO  ;  1  <  i  <  n}  the  eigenvalues  of  the  spectral  density  matrix  _f(ai)  .  We  will 

first  analyze  the  prediction  game.  Then,  we  will  study  the  interpolation  game.  We 

first  define  two  measures  that  we  will  use  in  our  presentation. 

Let  A  be  some  set  in  the  Borel  field  B  .  We  will  denote: 

7T 

m(A)  :  The  Lebesque  measure  of  A 


1  (to)  :  The  indicator  function  of  A 
A 


Also,  in  the  description  of  the  class  F^  in  (B) ,  we  will  denote: 
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[-7V.T]  -  U  Ai 


‘k+1  =  ^ 


do)  =  c  -  ^  ^  c . 


We  also  use  the  known  identity: 


tr  |  f(od)  dOJ  = 


:  I  2  ^<“>1  dw  ; 

*4  i=l  -• 


;  A  e  B 


Due  to  the  identity  in  (29) ,  and  due  to  the  fact  that  the  error  e  (f)  is  a 

P  “ 

function  of  the  eigenvalues  of  jE(w)  only,  the  maximization  of  e^(f)  on  F^  does  not 
involve  explicitly  the  eigenvectors  of  the  spectral  density  matrices  _f  (to)  .  Indeed, 
the  solution  of  the  prediction  game  on  F^xS^  can  be  formalized  as  an  optimization 
problem,  with  constraints  of  the  form  as  in  (29).  We  express  the  constraint  optimiza¬ 
tion  problem  and  its  solution  in  a  theorem. 

Theorem  5 

The  supremum  of  e^(f)  on  F^  in  (B)  corresponds  to  the  solution  of  the  following 
optimization  problem: 


sup  I  £og  [  II  X^(w)]  do) 


subject  to: 


j  £  X1(w)J  dcj  =  c^  ;  j=l . k+1 

A,  1 —  1 


;  where  (c ^ ,  Aj)  ;  1  <  j  <  k+1  the  pairs  given  in  the  description  of  the  class  Fr 


and  in  (28) 
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The  unique  solution  of  the  above  optimization  problem  is  represented  by  an 

it  if 

n-tuple  (A-j,  (jj)  , . . .  ,Xn(w))  of  eigenvalues,  such  that: 

k+1 

V  i  ;  A*(<o)  =  A*(w)  =  n-1  V  c.  1  (u)  m_1(A.)  (30) 

1  '  j  A.  j 

j=l  J 

;  where  1^(0))  and  m(A)  are  given  by  (27) . 

Proof 

The  exitence  of  a  unique  scrution  is  guaranteed  by  theorem  1.  It  is  thus 
sufficient  to  prove  that: 

r  r 

log  [A  (u>)]n  du  >  log  [  II  A1(u)]  d0J  ;  V  (X1(ta»)}  in  FQ  (31) 

J-1T  1_1 


it 

;  where  A  (cj)  is  given  by  (30)  . 

The  eigenvalues  (A^(u>)}  are  positive.  We  will  use  the  inequality  between  arithmetic 
and  geometric  means  for  positive  numbers,  as  well  as  the  generalized  form  of 
inequality,  known  as  Jensen's  inequality.  We  thus  obtain: 


-IT  /  k+1 

<e°8j  2 

J-TT  (  j=l 


-1 

n 


m‘1(V  b 


|*ir 

da)  ■  £og[A*(u>)]n  do) 

J-tt 


(32) 
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The  sequence  of  equalities  and  inequalities  in  (32)  hold  with  strict  equality  every¬ 
where,  if  and  only  if  A^(to)  =  A  (w) ;  V  i.  The  proof  is  now  complete. 


We  exhibit  the  solution  in  (30)  graphically,  in  figure  3.  We  will  complete 
the  coverage  of  the  prediction  game  in  the  class  F^  with  some  somments. 

Comments 

The  solution  in  theorem  5  implies  no  restrictions  on  the  eigenvectors  of  the 
spectral  density  matrices  fXw)  in  the  class  F  .  Indeed,  the  class  F^  imposes 
quantiled  energy  restrictions  only.  It  leaves  the  projections  of  the  spectral 
density  matrices  unrestricted.  The  solution  in  (30)  gives  spectra  that  are  piece- 
wise  constant  functions  in  [-TT,7f],  This  satisfies  our  intuition,  since  we  expect 
the  "f latest"  possible  spectra,  as  the  solution  to  the  prediction  game,  for  any 
convex  and  closed  class  of  measures.  Indeed,  the  prediction  error  has  entropy 
characteristics . 


We  now  consider  the  interpolation  game  on  F^xS^.  As  concluded  from  theorem  1, 
it  is  equivalent  to  search  for  the  supremum  of  the  error  e^(_f)  in  (15)  on  the  class 
Fq.  Since  the  error  e^(f)  involves  explicitly  both  the  eigenvalues  and  the  eigen¬ 
vectors  of  the  spectral  density  matrix  _f  (co) ,  we  will  analyze  the  lower  bound  bt  (f) 
in  (26)  instead  (as  we  did  in  section  3) .  As  in  section  3,  the  function  b  (jf)  is 

strictly  concave  on  F  .  We  will  thus  search  for  the  unique  supremum  of  b  (f)  on  F  . 

Q  L  Q 

This  search  can  be  formalized  as  a  constraint  optimization  problem.  We  state  the 
optimization  problem  and  its  solution  in  a  lemma. 

Lemma  3 

The  supremum  of  the  function  bL(f)  in  (26)  on  F^  corresponds  to  the  solution  of 
the  following  constraint  optimization  problem: 
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inf 


[£ 

-i  =  1 


x>>] 


do) 


;  where  (c^ ,  A..)  as  in  theorem  5. 

The  unique  solution  of  the  above  optimization  problem  is  identical  to  the 
solution  (30) in  theorem  5.  Furthermore,  at  this  solution  the  values  of  b  (f)  and 

la 

e  (f)  coincide. 

Proof 

We  follow  similar  approach  as  in  the  proof  of  theorem  5.  Again,  since  the 
•niqueness  of  the  solution  is  established  in  advance,  it  is  sufficient  to  show  that: 


/•TT 


n[X*  ((jj)  ]_1  dw  < 


-IT 


[£ 

-TT  i=l 


dco 


VtX^U))} 


in  F 


(33) 


;  where  X  (w)  the  solution  in  theorem  5. 

We  will  use  the  following  inequalities  on  positive  numbers  and  functions: 


N 


£ 


(34) 


(35) 


The  inequality  in  (34)  represents  a  relationship  between  the  arithmetic  and  the 
harmonic  mean  of  positive  numbers.  It  is  satisfied  with  equality  iff  =  a  ;  V  i. 
The  inequality  in  (35)  in  Jensen's  inequality,  where  m(A)  is  the  Lebesque  measure  of 


the  set  A.  Expression  (35)  is  satisfied  with  equality  iff  f(x)  is  constant  on  A. 
Using  the  inequalities  in  (34)  and  (35)  we  obtain: 


'•Ti  n 

k+1 

f  n  k+1 

r  n  ! 

[E  da) 

-£ 

[E  xi1<(j)Jdu- E 

n2  1  f  Xi(«o)J  dto  > 

J- n  i=l 

J=1 

4.  i=i  j=i 

4.  i=l 

J 

1  m2(Aj)  (36) 

But  for  the  solution  X  (to)  in  theorem  5,  we  obtain: 


■*ir 

n[X*(co)  ] 

J-7T 


/•TT 


dto  =  n 


k+1 


-■‘[L 


-IT 


j  =  l 


c.  m  X(A.) 
J  3 


dto  = 


fTT  k+1 

■  [E 


k+1 

=  n2£  cj-1m2(Aj) 

j=l 


m  1(Ai) 


1*  (“> 


dto 


(37) 


From  (36)  and  (37)  we  conclude  directly  that  (33)  is  satisfied.  Furthermore,  the 
inequalities  in  (36)  are  strict  equalities  iff  X^(to)  =  X*(to)  ;  V  i.  Thus,  the  left 
part  of  (33)  is  attained.  Finally,  as  in  the  proof  of  theorem  4,  if  jf_*  is  some 
spectral  density  matrix  with  eigenvalues  X^(to)  =  X*(to)  ;  V  i,  we  also  have 
e^Cf*)  *  b^(f*) .  The  proof  is  now  complete. 


We  will  conclude  this  section  with  some  comments  on  the  interpolation  game  on 
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Comments 


As  in  section  3,  the  involvement  of  eigenvectors  in  the  error  expression  e^(f) 
prevented  us  from  finding  a  general  solution  of  the  interpolation  game  on  F^xS^. 
Instead,  we  maximized  a  lower  bound  on  the  saddle  value  of  the  game.  The  supreaum 
of  this  lower  bound  b  (f)  is  equal  to  the  error  e  (f) ,  at  the  f  value  that  satisfies 

L  i  —  ' — 

the  supremum  sup  b  (f) .  If  the  spectral  density  matrices  in  F„  have  identical 

feFQ  L  “  Q 

constant  eigenvectors,  the  lower  bound  bT (f)  coincides  with  the  error  e.(f)  for  all 

l.  —  i  — 

_f.  Then,  the  solution  in  lemma  3  is  also  the  solution  of  the  interpolation  game. 


5.  Conclusions 

In  this  paper,  we  considered  the  prediction  and  interpolation  problems  for  vector 
stationary  processes  with  ill-specified  statistical  structures.  We  modeled  the 
uncertainty  in  the  statistical  description  of  the  processes  through  convex  and  closed 
families  of  vector  measures.  We  formalized  the  prediction  and  interpolation  problems 
as  statistical  games  with  saddle  point  solutions,  and  we  considered  two  different 
families  of  vector  measures.  The  first  such  family  represents  a  linear  contamination 
of  a  nominal  measure,  and  it  includes  an  energy  constraint.  We  provided  two  solutions 
of  the  prediction  game  within  this  family.  One  of  the  solutions  is  the  most  robust, 
while  the  second  solution  is  the  least  robust.  They  are  both  represented  by  specific 
choices  of  the  eigenvalues  of  spectral  density  matrices.  Within  the  same  family,  we 
analyzed  a  lower  bound  on  the  solution  of  the  interpolation  game,  and  we  found  the 
conditions  under  which  this  lower  bound  is  attained. 

The  second  family  of  vector  measures  we  considered  is  represented  by  fixed 
energy,  on  a  finite  number  of  prespecified  frequency  quantiles.  The  solution  of  the 
prediction  game  is  then  provided  by  identical  eigenvalues  such  that  each  is  piece-wise 
constant.  For  the  interpolation  game,  we  analyzed  a  lower  bound  instead,  and  we  found 
again  the  conditions  under  which  this  lower  bound  is  attained. 
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All  the  derived  solutions  for  the  prediction  game  correspond  to  the  eigenvalues 
with  the  "f latest"  possible  tails.  Equivalently,  these  solutions  correspond  to  the 
measures  with  the  most  evenly  spread  energy.  This  is  intuitively  satisfactory,  since 
the  prediction  error  has  entropy  characteristics  in  the  frequency  domain,  and  the 
most  even  spreading  results  in  entropy  maximization. 
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7.  Appendix 

Proof  of  Theorem  1 

We  will  prove  the  theorem  through  a  sequence  of  lemmas. 

Lemma  A.l 

Let  A  be  an  nxn  symmetric  and  positive  definite  matrix.  Then, 

i)  f  (A)  =  tr  -Cog  A 

is  strictly  concave  in  A 

ii)  h(A)  =  tr  A  ^ 

is  strictly  convex  in  A 

Proof 

Let  M  be  the  set  of  all  nxn  symmetric  and  positive  definite  matrices, 
convex  and  closed.  Thus,  if  A^  e  M,  A2  £  M,  then  [a  A^  +  (1-a)  A21  E  M  ;  V 
0  <  a  <  i. 

To  prove  the  lemma,  it  suffices  to  show  that: 

a2 

f  (a  AL  +  (1-a)  A2)  <  0  ;  V  A^  k2  e  M,  A^  4  A? 

9a 
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h  (a  Ax  +  (1-a)  A2)  >  0  ;  V  A^  A2  e  M,  Ax  +  A? 

9a  x 

But,  we  easily  obtain: 

2  2 

i)  — j -  f  (aA^  +  (1-a)  =  tr  — 2  Cog  [a  A^  +  (1-a)  A21  * 

9a  9a 

-  tr  [[a  Ax  +  (1-a)  A 2]-1  [A^  -  A2]J 
=  -  tr  [a  A1  +  (1-a)  A2]-2  (Aj  -  A21  [Aj^  -  A21  - 

n 

*>  -  tr  [Aj  -  A23T  [a  At  +  (1-a)  Aj]-2  [A1  -  A23  *  -  ^  c^  [a  Aj^ 

i-1 


Ai  is 
a  : 


+  (1-c)  A2]'2  c£ 


(A.l) 


33 


;  where  the  ith  column  of  the  symmetric  matrix  -  A^. 

Since  a  +  (1-a)  A,;  is  positive  definite,  so  will  be  the  matrix 

[a  A^  +  (1-a)  A23  Also,  if  A^  ¥  A2  some  columns  c^  will  be  nonzero. 
Thus,  the  expression  in  (A.l)  is  strictly  negative. 


ii)  — y  h  (a  A.  +  (1-a)  A2)  =  tr  [a  Aj^  +  (1-a)  A2]  1 

3a  3a 


-  tr  [a  Ax  +  (1-a)  A21  2  [A1  -  A2] 


2  tr  [a  A1  +  (1-a)  A2]  J  [Ax  -  A2]  [Ax  -  A2]  - 
2  tr  [A1  -  A21T  [a  Ax  +  (1-a)  A2]-3  [Ax  -  Ag]  (A. 2) 


If  a  A^  +  (1-a)  A2  is  a  positive  definite  matrix,  so  is  [a  A^  +  (1-a)  A21 
As  in  i) ,  for  A^  ¥  A2  the  expression  in  (A. 2)  is  strictly  positive. 


Lemma  A. 2 


The  errors  e  (y)  and  e^(y)  are  strictly  concave  in  F.  Thus,  unique  y*  and  y* 


exist  that  satisfy  the  suprema  in  (10)  and  (11). 


Proof 


i)  The  strict  concavity  of  e^(y)  is  directly  due  to  part  i)  of  lemma  A.l  and 
the  fact  that  y^  e  F,  y2  e  F  -*  [e  y^  +  (1-e)  y2l  e  F  ;  Ve:  0  <  e  <  1,  and 


-ey1+(l-e)y2 


(w)  =  e  f  ,  (10)  +  (1-e)  f  (w). 


ii)  To  prove  the  strict  concavity  of  e^(y) ,  we  select  y^,  y2  e  F,  and 

e  :  0  <  e  <  1.  Then,  [e  y^^  +  (1-e)  y2l  e  F,  and  we  can  write  from  (7): 


']  eA(e  yx  +  (l-e)  y2)  =  trM  (e  (w)  +  (1-e)  f^(w)) 


-1  do,]"1 


(A.  3) 
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3  -I  r  3  i 

Using  the  identity  —  A  (a)  =  -A~  (a)  I  ^  A(a)l  A-  (a)  for  matrices  and 
differentiating  expression  (A. 3)  twice  with  respect  to  e,  we  find: 


[4tt2]  1  e .  (e  y.  +  (1-e)  y7) 
3e  11  z 


=  -2tr 


Kb-1  g  b-1  b  b_1  +  f  b'1  b  b_1[|*  b_1]  b_1  b  b_1\a  1 

TT  J~ir  ./— TT  TT  ' 


(A. 4) 


;  where: 


B  =  e  f  (us)  +  (1-e)  f  (us) 


A  =  |  [e  f  (us)  +  (1-e)  f  (us)]'1  du 
-*  Ul  M2 

“-If6 

The  terms  in  (A. 4)  are  positive  definite,  thus  the  total  expression  is 
negative. 

Lemma  A. 3 

ic  is  is 

The  measures  yp  and  y^  of  lemma  A. 2  and  the  corresponding  predictor  g  and 
interpolator  g^  in  theorem  1  satisfy  respectively  the  games  in  (8)  and  (9) . 

Proof 

Let  us  consider  the  error  expressions  ep(y,  g^)  and  e^y,  g^)  given  by  (4)  and 

(5)  respectively.  Both  e^(y,  g^)  and  e^y,  g^,)  are  linear  in  y.  Also,  since  F  is 

a  closed  family,  the  suprema  sup  e  (y,  e  ),  and  sup  e. (y,  g.)  exist  and  are  attained 

yEF  p  T*  ycF  1  1 

in  F.  For  given  in  Sp  and  given  g^  in  S^,  let: 


“f  V-  V  ’  eP(\’  V 

jgp  el(u>  ■  ei(V  *i> 
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Then,  due  to  lemma  A. 2,  we  obtain: 


inf  sup  e  (y,g)  =>  Inf  e  (y  ,g)  <  Inf  e  (y  ,g°)  =  e  (y  )  <  e  (y*) 

VS.veF  P  ^  P  "tfeS.  P  S  P  ~  P  P 


«P 


(A.  5) 


inf  su 


inf  e  (y  <  inf  e  (y  ,e  )  =  e  'u  )  <  e  (y  ) 

1  SLj  —  _o<-c  i  gj  — i  i  1  1 


Ei^i  PeF  1  '"1  £1^i  &±  £^eS1  *i  _1 


But  by  definition: 

e  (y*)  =  sup  inf  e  (y,g  ) 

P  P  P  ^ 

(A.  6) 

e. (yt)  =  sup  inf  e  (y,g  ) 

1  1  iieF^eS,  1  1 


Thus,  from  (A. 5)  and  (A. 6)  we  obtain: 


inf  sup  e  (y,g  )  <  sup  inf  e  <V,Z  ) 
ye F  ^  yeF  ^ 

(A.  7) 

inf  sup  e  (y.jjJ  <  sup  inf  e  (y,£.) 

IL^yeF  1  ^  “yeF£le5i  1  1 


But  it  is  always  true  that  sup  inf  f(x,y)  <  inf  sup  f(x,y)  for  any  function 

X  y  y  X 

f (x,y) .  Due  to  that,  expressions  (A. 7)  roust  hold  with  equality.  Thus,  the  games 
have  unique  solutions  given  by  the  statement  of  the  lemma. 


